{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Gathering\n",
    "\n",
    "This recipe shows how we will be accessing the datasets necessary for the rest of the book.\n",
    "\n",
    "We start by loading the necessary libraries and resetting the computational graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\Anaconda3_5_0_0\\lib\\site-packages\\h5py\\__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.framework import ops\n",
    "ops.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Iris Dataset (R. Fisher / Scikit-Learn)\n",
    "\n",
    "One of the most frequently used ML datasets is the iris flower dataset.  We will use the easy import tool, `datasets` from scikit-learn.  You can read more about it here: http://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "150\n",
      "150\n",
      "[5.1 3.5 1.4 0.2]\n",
      "{0, 1, 2}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "\n",
    "iris = load_iris()\n",
    "print(len(iris.data))\n",
    "print(len(iris.target))\n",
    "print(iris.data[0])\n",
    "print(set(iris.target))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Low Birthrate Dataset (Hosted on Github)\n",
    "\n",
    "The 'Low Birthrate Dataset' is a dataset from a famous study by Hosmer and Lemeshow in 1989 called, \"Low Infant Birth Weight Risk Factor Study\".  It is a very commonly used academic dataset mostly for logistic regression.  We will host this dataset on the public Github here:\n",
    "https://github.com/nfmcclure/tensorflow_cookbook/raw/master/01_Introduction/07_Working_with_Data_Sources/birthweight_data/birthweight.dat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "189\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "birthdata_url = 'https://github.com/nfmcclure/tensorflow_cookbook/raw/master/01_Introduction/07_Working_with_Data_Sources/birthweight_data/birthweight.dat'\n",
    "birth_file = requests.get(birthdata_url)\n",
    "birth_data = birth_file.text.split('\\r\\n')\n",
    "birth_header = birth_data[0].split('\\t')\n",
    "birth_data = [[float(x) for x in y.split('\\t') if len(x)>=1] for y in birth_data[1:] if len(y)>=1]\n",
    "print(len(birth_data))\n",
    "print(len(birth_data[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Housing Price Dataset (UCI)\n",
    "\n",
    "We will also use a housing price dataset from the University of California at Irvine (UCI) Machine Learning Database Repository. It is a great regression dataset to use.  You can read more about it here:\n",
    "https://archive.ics.uci.edu/ml/datasets/Housing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "506\n",
      "14\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "housing_url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data'\n",
    "housing_header = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV']\n",
    "housing_file = requests.get(housing_url)\n",
    "housing_data = [[float(x) for x in y.split(' ') if len(x)>=1] for y in housing_file.text.split('\\n') if len(y)>=1]\n",
    "print(len(housing_data))\n",
    "print(len(housing_data[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MNIST Handwriting Dataset (Yann LeCun)\n",
    "\n",
    "The MNIST Handwritten digit picture dataset is the `Hello World` of image recognition. The famous scientist and researcher, Yann LeCun, hosts it on his webpage here, http://yann.lecun.com/exdb/mnist/ .  But because it is so commonly used, many libraries, including TensorFlow, host it internally.  We will use TensorFlow to access this data as follows.\n",
    "\n",
    "If you haven't downloaded this before, please wait a bit while it downloads"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-6-383a9e7997d8>:3: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From E:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From E:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\base.py:252: _internal_retry.<locals>.wrap.<locals>.wrapped_fn (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use urllib or similar directly.\n"
     ]
    },
    {
     "ename": "URLError",
     "evalue": "<urlopen error [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。>",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m----------\u001b[0m",
      "\u001b[1;31mTimeoutError\u001b[0mTraceback (most recent call last)",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36mdo_open\u001b[1;34m(self, http_class, req, **http_conn_args)\u001b[0m\n\u001b[0;32m   1317\u001b[0m                 h.request(req.get_method(), req.selector, req.data, headers,\n\u001b[1;32m-> 1318\u001b[1;33m                           encode_chunked=req.has_header('Transfer-encoding'))\n\u001b[0m\u001b[0;32m   1319\u001b[0m             \u001b[1;32mexcept\u001b[0m \u001b[0mOSError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;31m# timeout error\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\http\\client.py\u001b[0m in \u001b[0;36mrequest\u001b[1;34m(self, method, url, body, headers, encode_chunked)\u001b[0m\n\u001b[0;32m   1238\u001b[0m         \u001b[1;34m\"\"\"Send a complete request to the server.\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1239\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_send_request\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmethod\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mbody\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mencode_chunked\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1240\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\http\\client.py\u001b[0m in \u001b[0;36m_send_request\u001b[1;34m(self, method, url, body, headers, encode_chunked)\u001b[0m\n\u001b[0;32m   1284\u001b[0m             \u001b[0mbody\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_encode\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbody\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'body'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1285\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mendheaders\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbody\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mencode_chunked\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mencode_chunked\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1286\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\http\\client.py\u001b[0m in \u001b[0;36mendheaders\u001b[1;34m(self, message_body, encode_chunked)\u001b[0m\n\u001b[0;32m   1233\u001b[0m             \u001b[1;32mraise\u001b[0m \u001b[0mCannotSendHeader\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1234\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_send_output\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmessage_body\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mencode_chunked\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mencode_chunked\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1235\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\http\\client.py\u001b[0m in \u001b[0;36m_send_output\u001b[1;34m(self, message_body, encode_chunked)\u001b[0m\n\u001b[0;32m   1025\u001b[0m         \u001b[1;32mdel\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_buffer\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1026\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1027\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\http\\client.py\u001b[0m in \u001b[0;36msend\u001b[1;34m(self, data)\u001b[0m\n\u001b[0;32m    963\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mauto_open\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 964\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mconnect\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    965\u001b[0m             \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\http\\client.py\u001b[0m in \u001b[0;36mconnect\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m   1391\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1392\u001b[1;33m             \u001b[0msuper\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mconnect\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1393\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\http\\client.py\u001b[0m in \u001b[0;36mconnect\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    935\u001b[0m         self.sock = self._create_connection(\n\u001b[1;32m--> 936\u001b[1;33m             (self.host,self.port), self.timeout, self.source_address)\n\u001b[0m\u001b[0;32m    937\u001b[0m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msock\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msetsockopt\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msocket\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mIPPROTO_TCP\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msocket\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTCP_NODELAY\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\socket.py\u001b[0m in \u001b[0;36mcreate_connection\u001b[1;34m(address, timeout, source_address)\u001b[0m\n\u001b[0;32m    721\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0merr\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 722\u001b[1;33m         \u001b[1;32mraise\u001b[0m \u001b[0merr\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    723\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\socket.py\u001b[0m in \u001b[0;36mcreate_connection\u001b[1;34m(address, timeout, source_address)\u001b[0m\n\u001b[0;32m    712\u001b[0m                 \u001b[0msock\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbind\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msource_address\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 713\u001b[1;33m             \u001b[0msock\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mconnect\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msa\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    714\u001b[0m             \u001b[1;32mreturn\u001b[0m \u001b[0msock\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mTimeoutError\u001b[0m: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mURLError\u001b[0mTraceback (most recent call last)",
      "\u001b[1;32m<ipython-input-6-383a9e7997d8>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;32mfrom\u001b[0m \u001b[0mtensorflow\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mexamples\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtutorials\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmnist\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0minput_data\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[0mmnist\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0minput_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mread_data_sets\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"MNIST_data/\"\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mone_hot\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mTrue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmnist\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtrain\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mimages\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmnist\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtest\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mimages\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\u001b[0m in \u001b[0;36mnew_func\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    248\u001b[0m               \u001b[1;34m'in a future version'\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mdate\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;34m'after %s'\u001b[0m \u001b[1;33m%\u001b[0m \u001b[0mdate\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    249\u001b[0m               instructions)\n\u001b[1;32m--> 250\u001b[1;33m       \u001b[1;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    251\u001b[0m     return tf_decorator.make_decorator(\n\u001b[0;32m    252\u001b[0m         \u001b[0mfunc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnew_func\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'deprecated'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py\u001b[0m in \u001b[0;36mread_data_sets\u001b[1;34m(train_dir, fake_data, one_hot, dtype, reshape, validation_size, seed, source_url)\u001b[0m\n\u001b[0;32m    258\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    259\u001b[0m   local_file = base.maybe_download(TRAIN_IMAGES, train_dir,\n\u001b[1;32m--> 260\u001b[1;33m                                    source_url + TRAIN_IMAGES)\n\u001b[0m\u001b[0;32m    261\u001b[0m   \u001b[1;32mwith\u001b[0m \u001b[0mgfile\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mOpen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlocal_file\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'rb'\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    262\u001b[0m     \u001b[0mtrain_images\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mextract_images\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mf\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\u001b[0m in \u001b[0;36mnew_func\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    248\u001b[0m               \u001b[1;34m'in a future version'\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mdate\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;34m'after %s'\u001b[0m \u001b[1;33m%\u001b[0m \u001b[0mdate\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    249\u001b[0m               instructions)\n\u001b[1;32m--> 250\u001b[1;33m       \u001b[1;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    251\u001b[0m     return tf_decorator.make_decorator(\n\u001b[0;32m    252\u001b[0m         \u001b[0mfunc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnew_func\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'deprecated'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\base.py\u001b[0m in \u001b[0;36mmaybe_download\u001b[1;34m(filename, work_directory, source_url)\u001b[0m\n\u001b[0;32m    250\u001b[0m   \u001b[0mfilepath\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mos\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mwork_directory\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfilename\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    251\u001b[0m   \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mgfile\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mExists\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfilepath\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 252\u001b[1;33m     \u001b[0mtemp_file_name\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0m_\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0murlretrieve_with_retry\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msource_url\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    253\u001b[0m     \u001b[0mgfile\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mCopy\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtemp_file_name\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfilepath\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    254\u001b[0m     \u001b[1;32mwith\u001b[0m \u001b[0mgfile\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mGFile\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfilepath\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\u001b[0m in \u001b[0;36mnew_func\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    248\u001b[0m               \u001b[1;34m'in a future version'\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mdate\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;34m'after %s'\u001b[0m \u001b[1;33m%\u001b[0m \u001b[0mdate\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    249\u001b[0m               instructions)\n\u001b[1;32m--> 250\u001b[1;33m       \u001b[1;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    251\u001b[0m     return tf_decorator.make_decorator(\n\u001b[0;32m    252\u001b[0m         \u001b[0mfunc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnew_func\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'deprecated'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\base.py\u001b[0m in \u001b[0;36mwrapped_fn\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    203\u001b[0m       \u001b[1;32mfor\u001b[0m \u001b[0mdelay\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mdelays\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    204\u001b[0m         \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 205\u001b[1;33m           \u001b[1;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    206\u001b[0m         \u001b[1;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m  \u001b[1;31m# pylint: disable=broad-except\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    207\u001b[0m           \u001b[1;32mif\u001b[0m \u001b[0mis_retriable\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\base.py\u001b[0m in \u001b[0;36murlretrieve_with_retry\u001b[1;34m(url, filename)\u001b[0m\n\u001b[0;32m    231\u001b[0m \u001b[1;33m@\u001b[0m\u001b[0m_internal_retry\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0minitial_delay\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m1.0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmax_delay\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m16.0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mis_retriable\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0m_is_retriable\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    232\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0murlretrieve_with_retry\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfilename\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 233\u001b[1;33m   \u001b[1;32mreturn\u001b[0m \u001b[0murllib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrequest\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0murlretrieve\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfilename\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    234\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    235\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36murlretrieve\u001b[1;34m(url, filename, reporthook, data)\u001b[0m\n\u001b[0;32m    246\u001b[0m     \u001b[0murl_type\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpath\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0msplittype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murl\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    247\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 248\u001b[1;33m     \u001b[1;32mwith\u001b[0m \u001b[0mcontextlib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclosing\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murlopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0mfp\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    249\u001b[0m         \u001b[0mheaders\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mfp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    250\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36murlopen\u001b[1;34m(url, data, timeout, cafile, capath, cadefault, context)\u001b[0m\n\u001b[0;32m    221\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    222\u001b[0m         \u001b[0mopener\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_opener\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 223\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0mopener\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    224\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    225\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0minstall_opener\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mopener\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36mopen\u001b[1;34m(self, fullurl, data, timeout)\u001b[0m\n\u001b[0;32m    524\u001b[0m             \u001b[0mreq\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmeth\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mreq\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    525\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 526\u001b[1;33m         \u001b[0mresponse\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_open\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mreq\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    527\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    528\u001b[0m         \u001b[1;31m# post-process response\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36m_open\u001b[1;34m(self, req, data)\u001b[0m\n\u001b[0;32m    542\u001b[0m         \u001b[0mprotocol\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreq\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtype\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    543\u001b[0m         result = self._call_chain(self.handle_open, protocol, protocol +\n\u001b[1;32m--> 544\u001b[1;33m                                   '_open', req)\n\u001b[0m\u001b[0;32m    545\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mresult\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    546\u001b[0m             \u001b[1;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36m_call_chain\u001b[1;34m(self, chain, kind, meth_name, *args)\u001b[0m\n\u001b[0;32m    502\u001b[0m         \u001b[1;32mfor\u001b[0m \u001b[0mhandler\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mhandlers\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    503\u001b[0m             \u001b[0mfunc\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mhandler\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmeth_name\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 504\u001b[1;33m             \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    505\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mresult\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    506\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36mhttps_open\u001b[1;34m(self, req)\u001b[0m\n\u001b[0;32m   1359\u001b[0m         \u001b[1;32mdef\u001b[0m \u001b[0mhttps_open\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mreq\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1360\u001b[0m             return self.do_open(http.client.HTTPSConnection, req,\n\u001b[1;32m-> 1361\u001b[1;33m                 context=self._context, check_hostname=self._check_hostname)\n\u001b[0m\u001b[0;32m   1362\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1363\u001b[0m         \u001b[0mhttps_request\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mAbstractHTTPHandler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdo_request_\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mE:\\Anaconda3_5_0_0\\lib\\urllib\\request.py\u001b[0m in \u001b[0;36mdo_open\u001b[1;34m(self, http_class, req, **http_conn_args)\u001b[0m\n\u001b[0;32m   1318\u001b[0m                           encode_chunked=req.has_header('Transfer-encoding'))\n\u001b[0;32m   1319\u001b[0m             \u001b[1;32mexcept\u001b[0m \u001b[0mOSError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;31m# timeout error\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1320\u001b[1;33m                 \u001b[1;32mraise\u001b[0m \u001b[0mURLError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0merr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1321\u001b[0m             \u001b[0mr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mh\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgetresponse\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1322\u001b[0m         \u001b[1;32mexcept\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mURLError\u001b[0m: <urlopen error [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。>"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)\n",
    "print(len(mnist.train.images))\n",
    "print(len(mnist.test.images))\n",
    "print(len(mnist.validation.images))\n",
    "print(mnist.train.labels[1,:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CIFAR-10 Data\n",
    "\n",
    "The CIFAR-10 data ( https://www.cs.toronto.edu/~kriz/cifar.html ) contains 60,000 32x32 color images of 10 classes collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.  Alex Krizhevsky maintains the page referenced here.  This is such a common dataset, that there are built in functions in TensorFlow to access this data (the keras wrapper has these commands). Note that the keras wrapper for these functions automatically splits the images into a 50,000 training set and a 10,000 test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n",
      "170500096/170498071 [==============================] - 151s 1us/step\n"
     ]
    }
   ],
   "source": [
    "from PIL import Image\n",
    "# Running this command requires an internet connection and a few minutes to download all the images.\n",
    "(X_train, y_train), (X_test, y_test) = tf.contrib.keras.datasets.cifar10.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ten categories are (in order):\n",
    "\n",
    "<ol start=\"0\">\n",
    "  <li>Airplane</li>\n",
    "  <li>Automobile</li>\n",
    "  <li>Bird</li>\n",
    "  <li>Car</li>\n",
    "  <li>Deer</li>\n",
    "  <li>Dog</li>\n",
    "  <li>Frog</li>\n",
    "  <li>Horse</li>\n",
    "  <li>Ship</li>\n",
    "  <li>Truck</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50000, 32, 32, 3)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50000, 1)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([6], dtype=uint8)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[0,] # this is a frog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x15af032dcf8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH3FJREFUeJztnVuMXNd1pv9Vt67qezf7QrJJiRJ1GcmxRMmMIEiZjB3P\nBIoRRDaQZOwHQw9GGAQxEAPJg+AAYw8wD/ZgbMMPAw/okRJl4PFlfImFQJjEEWwIiQNFlCXrHomi\nKLHJVrPJ7mZ3dVXXdc1DlyZUa/+bJTZZTWn/H0B0ca/a56zaddY5VeevtZa5O4QQ6ZHZbgeEENuD\ngl+IRFHwC5EoCn4hEkXBL0SiKPiFSBQFvxCJouAXIlEU/EIkSm4rk83sHgBfB5AF8D/d/Uux5+fz\nee8rFoO2VqtF52UQ/hVi1vi+Cjl+XstHbLlsltrMwjs0i5xDIz42m/w1x353mY35SH6x2fY231eb\n780ykRcQod0Ov7aY79HtRfy3yCIzWybiRzbD3092DABAO/JrWY8dCGxOdHthFpdXUa6sd7Wziw5+\nM8sC+O8A/gOAWQBPmNnD7v4Cm9NXLOLA7R8K2paXF+m++jLhN368wBfnqh391DY5PkBtE6OD1FbI\n5oPjub4SnYMsX+LFpWVqqzf5axsbHaG2TKsRHK/VanTO+vo6tRVL4ZM1ALTAT16Vajk4PjI6TOfA\n+fbqtTq1ZRF+XwB+shka5O/zwAA/PvJ5vh7ViI8eu0BkwsdI7DU3PRzfX37gB3w/m3fb9TPfyR0A\njrr7MXevA/gOgHu3sD0hRA/ZSvDPADhx3v9nO2NCiPcAW/nOH/rc8Y7PqmZ2CMAhAOjr69vC7oQQ\nl5KtXPlnAew97/97AJza/CR3P+zuB939YC7Pv5sJIXrLVoL/CQDXm9k1ZlYA8EkAD18at4QQl5uL\n/tjv7k0z+yyAv8WG1Peguz8fm7O+vo7nXwg/ZfnMGTpvnNxgtR38zutEa4jarDRFbWttrjqUW+E7\n8G4FOqeyzu/YVqr8DnyjxaWtMxGNs5gL+9hs8u1lyd1mIP5VrbK+Rm3Ndvh12/oOOicTUQEbEbWi\nlOPHQZncMV9sNemc/n5+t98y/NOrETUIABCRDyvrYYWm2QiPA0A2F35fGutV7sMmtqTzu/sjAB7Z\nyjaEENuDfuEnRKIo+IVIFAW/EImi4BciURT8QiTKlu72v1syAEo5IlNFfvx3NZH09k3zBJepyXFq\nK8WknEjWVrUWToBZb3AZyiPbK5QiCUGRxB5v8/2NjIcTmpoNvr1CnvsRSbZEtsDftFo9vFaNJl+P\n/sj2cgPcx2JkXtPCcmQmkiXYjGTgxTJJBwd4Mll5rUJtjWZY0oslVK6unAuOt2Nv2Obtd/1MIcT7\nCgW/EImi4BciURT8QiSKgl+IROnp3X4zR9HCCRVDQ9yVG2bGguM7SjwTJN/mpanKizzZptXm58Nq\nJex7huf1YDhSFiwXuUu9fG6Vz4u8a+ND4TvOqys8CaceSdCpkqQTIF6XbpCUwmrUeeJJpsVfWD6S\nYNQipcsAIEduz9dqfE4hz9/QTJsnBNXKS9QGkhQGAH3kMG62uSJxbi2s+LQi9Rg3oyu/EImi4Bci\nURT8QiSKgl+IRFHwC5EoCn4hEqWnUl/ODGN94V2WIlLOCEnqmBzmNdNapF0UgEifGSCbixSSI3XY\nau2I1BTR5XKR5JJWjUtinuXn7NOnw12AWg3+qlcrPOmk0uKy6GAp0n2nRtp1gb/mjHGZKtsX6ZSz\nxmXd/nzYx1ykFdZ6pO5itcGlvnakydpymfu4XAkfP2UiLQPAeiN8DNQjtRo3oyu/EImi4BciURT8\nQiSKgl+IRFHwC5EoCn4hEmVLUp+ZHQewig31rOnuB6M7yxomR8OSzVCeS2zFYtiWyXJppRSpj9do\nctmrHclUcw9LQPVIvb1WncuAbY9kzEUkNs/xrLPVejhDr9Xi61uJtAZrRmyra9z/k4thP/IZvr3h\nMl/7xpu8nVv1HJcqr5q4Ljg+NbWHzrGhcH08AKgtnaW2cplnR55b5VLfmXNhWff4Ce5HKxsO3Vqd\ny4ObuRQ6/0fcnb8zQogrEn3sFyJRthr8DuDvzOxJMzt0KRwSQvSGrX7sv9vdT5nZFICfmNlL7v7Y\n+U/onBQOAUAx8r1eCNFbtnTld/dTnb+nAfwIwB2B5xx294PufrCQ07cMIa4ULjoazWzAzIbeegzg\nNwE8d6kcE0JcXrbysX8awI867a1yAP63u//f2IR8Lovdk+HCjsMFLlEM9oelLYtIZYhkWFkkm65W\n5bJRhsiAO4Z427CBAZ6NtnKOiyQjwzxjbjVSVPP1k+Ftlmv8K1chkgg20x/JSszzzMPjZ8PZhTWP\nFF2NZPWNDA9R2103c4V5ZS4s63olsq8Jni1aq/D1KJf5tbQvz7e5d2f4tU1NTdM58yth6fDsy2/S\nOZu56OB392MAbr3Y+UKI7UVfwoVIFAW/EImi4BciURT8QiSKgl+IROltAc+sYXwonG2Xq4elIQDo\ny4fd7O8L96UDgFqVy2GNSL+10dFwX0AAcFL0sd7i59BGI1JccpD38Tu1EO7FBgCvvs6zvRZWw68t\nUgsSV0d6Hn783x6gtj27uP/ff/JYcPyfjnIpqtnmmYy5DJfmVpcXqK1SDq/j0BCX3tDi2YXFIp9X\nINmnANBvfF6zFX5zrtq7m84ZWgz3cnzmNb4Wm9GVX4hEUfALkSgKfiESRcEvRKIo+IVIlN7e7c/l\nMDW+I2irLvK74hkLu1kmbY4AoBqpZZazSD27SFsrdqasNvhd6tExnqBTb/E72MdmT1Hb4gr3kdX3\ny0ZafA0X+famcuG7ygBQXOSKxPXDO4Pjc+Pcj/nl09RWq/A1furll6ktQ9pXNQYircZGeEINMjxk\nRka4+jTUjrQHI3Uevb5C5+wjCXJ9+e6v57ryC5EoCn4hEkXBL0SiKPiFSBQFvxCJouAXIlF6LPXl\nMTYxGbSNDfL2WplMOClieWWJzmmslfn2WrF2XbygnZMEo8FBXqevAW578RiXqNZqvPVTsdjHbYWw\nj6UBLkONZbks+uTReWpr1vnhUxsJS32TY3w9DFx+azS5FFyp81qCa6RWX73JX7NFpNtINzfkM5FW\nb5lI7cJceB2bNS6lOpGJSe5ZEF35hUgUBb8QiaLgFyJRFPxCJIqCX4hEUfALkSgXlPrM7EEAvw3g\ntLv/SmdsHMB3AewDcBzA77s7193+dWsAke0s0s6I0Repp9aPcNYTAOQi57xMJlKPj8iAfSXeruvM\nmzwrrnKGL9m141wSq3HVC0Ui6d24f4bOyUQ22MzyNV6JSK25bLjO4FCBvy87xvZT2/7rr6K21954\ngtpeevlkcLyQi8hozmXiZpOHTIZkVAJAvsDXsd0OH1ftiK5oFj5OI0rkO+jmyv+XAO7ZNHY/gEfd\n/XoAj3b+L4R4D3HB4Hf3xwAsbhq+F8BDnccPAfj4JfZLCHGZudjv/NPuPgcAnb9Tl84lIUQvuOw3\n/MzskJkdMbMjq5XIl1UhRE+52OCfN7NdAND5S+svufthdz/o7geH+vlNLCFEb7nY4H8YwH2dx/cB\n+PGlcUcI0Su6kfq+DeDDACbMbBbAFwB8CcD3zOwzAN4A8Hvd7Kztjup6uFihNXhmFhDOwFpb4wUO\n6w1+Xmtm+CeQcoVLcyvENrOXL6M3+faunuDCzP7dXBqqrPN5MzfcGhwvOP/KtXSOF0ItjYYLrgIA\nzvJMtb07dwXHl9d4tuK1/+Z6ahse41mJw2M3UdvSQnj9l87xlmf5iByZcZ5R2WhHskV5sihajfDx\nHUkSpK3j3kVS34WD390/RUwffRf7EUJcYegXfkIkioJfiERR8AuRKAp+IRJFwS9EovS0gKfD0bKw\nHOItXlCRyRqlIi/6OTjEpaFTC1xWfG12gdpy+bAfhXneV299nm/v+iku5330w1z2evXk5lSLf2Vo\nJlwgdWJHuKAmAJxe4EU6R0cjsleb+18gBStPL4Sz7AAgV1ymtoXlOWo7Ocez8PL58HEwOsy1t2qV\nC2ae49dLi2hz7YgMmLHwPItkmEbaPHaNrvxCJIqCX4hEUfALkSgKfiESRcEvRKIo+IVIlJ5Kfdls\nBqOjg0FbM8elvnI5nJHmDS6fnFvlWVuvv8GlrXKZy0alYvhcOfcazy6cLvKijjMzV1Pb6O5rqC2/\nGkkRI0VN99x6B5/yJpffSk0uVbbAMwXX1sK2Xf1hKRIA6i3+umwgfNwAwJ6B3dQ2NBqWOFfPvknn\nnJ4/S20N4/Lmep0XBUWGa3MDfeEs03o1ImGSgqBGZMOgS10/UwjxvkLBL0SiKPiFSBQFvxCJouAX\nIlF6ere/3WpidTl8JzVX57Xu8qQ1EXgJOeSy3FgpcyVgbIgnsowOhO/KVpf43f6p3bwG3swt/47a\nnputU9vLR7ntrl3jwfHlZT5nen+47h8AZFChtnqNKwGjHr5zv3Ka30kv1XktwV3j4dcFAMstXlcv\nf8tYcLwaSRT6x0ceprbZE/w1ZyMtuWKNtFgeUSPWVq4RXiuWBBfcRtfPFEK8r1DwC5EoCn4hEkXB\nL0SiKPiFSBQFvxCJ0k27rgcB/DaA0+7+K52xLwL4AwBv6R6fd/dHutlhligerUgSgxOZJEPaeAFA\ny7jUt8QVJaysROq31cJy2a4RLg/+6kc+Qm17bryT2n74Fw9S285Ikku2Hq5PePLYq3x7195MbcUd\n11HbgHN5trIY7t1aaoelNwCoV7mseGaV20YneRLUjp37guPV8jCdk+EmtAo8mSlWw6/R4FKrNcMJ\nauY8ca3ZDIfupZb6/hLAPYHxr7n7gc6/rgJfCHHlcMHgd/fHAPBysUKI9yRb+c7/WTN7xsweNDP+\nWU4IcUVyscH/DQD7ARwAMAfgK+yJZnbIzI6Y2ZFyhX/vEUL0losKfnefd/eWu7cBfBMALRPj7ofd\n/aC7Hxzs51VthBC95aKC38x2nfffTwB47tK4I4ToFd1Ifd8G8GEAE2Y2C+ALAD5sZgcAOIDjAP6w\nm50ZACNKRItkKQG8bVGkcxK8GtlepATe+A7e5mtnf1havP3gDXTOTXdxOW/pNJc3+5o88/DaPXuo\nrU1e3M4pXjuvuc4l00okG7De5PMa1fCh1QKXKV89OUttzz53hNruupP7uGNnOKtyZTUsRQIA6fAF\nAJjYx2Xddqy9Vj0i2xEJ+dwCb19WWw072SbZlCEuGPzu/qnA8ANd70EIcUWiX/gJkSgKfiESRcEv\nRKIo+IVIFAW/EInS0wKe7kCbZDBVa1yiKJAstlyOF0zMZrj8c91O/mvkYomfD/ddvTc4fuuv8cy9\nXTfeQm1P/9NfUNtVe7mPOz/wQWorTO4Pjuf6R+icyjqXHKsrPHNv/tQJaluaD8t2rQbPzisNhQuk\nAsDEBH+vT5x6itqmd80Ex5uVSBZplbfdsrUlamt5OKMSAJxp3ABKfeHXVtjJX/NKH8l0fRcRrSu/\nEImi4BciURT8QiSKgl+IRFHwC5EoCn4hEqWnUp+ZIZ8N73IpUqCxtR6WNUr9JTonm+HSylQkc+/E\nHM+k2n97qJQhsOeD4fENuGTXWF2jtpEhLs1N3nCA2tZy4Z52zz/1BJ1Tq3I/Vlb4epw5+Qa1ZVth\nqbVY5IfczDVhWQ4AbrmBFxJtZnmmXT47Gh4v8KzP3Dov0ll5/SS1MRkbAJqRy2yZ9JXs38Ff1zTp\nAZnPd38915VfiERR8AuRKAp+IRJFwS9Eoij4hUiU3ib2tNuoVcN3Uvv7uCtWDN8NzWd4DTlvcVtp\nkLfy+p3/+DvUdtdvfTQ4PjwxTefMH3uR2rIR/5dXeQ2/heP/Qm2nVsN3nH/2139N5wyWeALJeo0n\nwOyc5orE8FD4TvVrszwZqB5Zj/Hd+6jthg9+iNrQ6gsOLy7zeoEVoi4BwFKV+2jOj+H1Kk9cK5MW\nW17mqsNNYRED7e67denKL0SqKPiFSBQFvxCJouAXIlEU/EIkioJfiETppl3XXgB/BWAngDaAw+7+\ndTMbB/BdAPuw0bLr992dFzgD4HC0ndTWa/OkCGuGZZKmR1pyRWqmFfuGqe3Ah7hs1JcPS2IvPM1r\nyC2depXaajUu5awuLVLbiaMvUFvZw8lO+Rbf12COS5/DRZ5cMjnGpb65+TeD481IW7bKKpcVT7zG\nk4iA56mlXA7XICzm+PHR7JuitrNNfuyUSrwGYf8QT0Ir5cJy5Gplhc5ptsOS47tQ+rq68jcB/Km7\n3wTgTgB/bGY3A7gfwKPufj2ARzv/F0K8R7hg8Lv7nLv/ovN4FcCLAGYA3Avgoc7THgLw8cvlpBDi\n0vOuvvOb2T4AtwF4HMC0u88BGycIAPyzkhDiiqPr4DezQQA/APA5d+dfRt4575CZHTGzI2tVXktf\nCNFbugp+M8tjI/C/5e4/7AzPm9mujn0XgGDDc3c/7O4H3f3gQKlwKXwWQlwCLhj8ZmYAHgDwort/\n9TzTwwDu6zy+D8CPL717QojLRTdZfXcD+DSAZ83s6c7Y5wF8CcD3zOwzAN4A8HsX3pRjQy18J+0m\n/0qQy4dr7rUiNdPq4NlX0yO8rt7fPvw31DY+HZaUpnaF23gBQL3Cs/Py+bDEAwCDA1xSymW4NDdA\n5MidU+GabwBQXeUKbSnLfTy7cIbaGvXwezNU5JJXvcylvleeOkJtcy+9TG21Jmmhledr2Iqt7x4u\nfWKAH8OZPi61FolsNwa+Vjd94JrgeKl4jM7ZzAWD393/AQDLcQznuAohrnj0Cz8hEkXBL0SiKPiF\nSBQFvxCJouAXIlF6WsATbmi3w8JBIZJZVsyR4ocZXmjRIy2c2nWeWXbmTDgbDQDKC2FbqcF/8NgG\nf13jY1x+G909SW3NVo3aTp4K++iRfK9Mhh8G9SaXTLPGC38OFMPyLEnQ3NhezBjJ0mzVuZyaIcfb\nSoXLm/U+Ig8CGNrN136txFubrba5DLi+Fr4G7xi+ls6ZINJtLt99SOvKL0SiKPiFSBQFvxCJouAX\nIlEU/EIkioJfiETprdQHQ8bCWWLFPp7B5CRDb6AUlpMAYGBogtoqDZ5htWOI1xzIET/q5+bpnHaG\nb6+S59LW9HQ4awsA2nUuG914y57g+M9/+iidU/cKteWNy6nVMp83PBTOSizk+CGXtUg/u3X+nr02\nx2W75eXwe1azNTpn8gZ+TZwZjWQlOn+vl87wtSqshyXTgZlIJmYlnDXZjqilm9GVX4hEUfALkSgK\nfiESRcEvRKIo+IVIlJ7e7c8YUMiFzzeVGk+YyJKWUe1IfblKgydnZPM8SaSvwO/m5vNhPwr9vG3V\nyDBPMHpzgasElZnwXXsAmNp7HbWdPB2uq/eBX72bzikvnKK2Yy/zVlhrZZ7IksuG139khNcmNFLf\nEQDmTnIf33g9ktjTF17/4WmuFE2OR3yMqA62yN/rsSUeajNT48HxPaP8GDj6QjiBq1blSWub0ZVf\niERR8AuRKAp+IRJFwS9Eoij4hUgUBb8QiXJBqc/M9gL4KwA7sdFr67C7f93MvgjgDwAsdJ76eXd/\nJLqznGF6Mny+aZw9S+dVW2EJaI3nZsAzvJVXLpJcMjzMkykKpBVWdY3X8CvFaqrVue3Iz39Obdfe\nyCXC2dmwBJSJ1Dvs7+O1+LIRObVU4tLWWjks9VWrXIJtRlq2DZa4H3fddgO1FUmCUTPLaxO2GjwJ\np3qCS32Z1SK1TfUPUdttN3wgPGd0ms55cu614HizwV/XZrrR+ZsA/tTdf2FmQwCeNLOfdGxfc/f/\n1vXehBBXDN306psDMNd5vGpmLwKYudyOCSEuL+/qO7+Z7QNwG4DHO0OfNbNnzOxBM+Otb4UQVxxd\nB7+ZDQL4AYDPufsKgG8A2A/gADY+GXyFzDtkZkfM7MhKhX+nE0L0lq6C38zy2Aj8b7n7DwHA3efd\nveXubQDfBHBHaK67H3b3g+5+cLifVzoRQvSWCwa/mRmABwC86O5fPW9813lP+wSA5y69e0KIy0U3\nd/vvBvBpAM+a2dOdsc8D+JSZHQDgAI4D+MMLbahQMFy1N3z1HzEukxw9EZZe5hd4dl69xaWhwUH+\nstcqPEOs1S4Hx7ORc+jiApcwV8tclllvcD+yzm1Dg+FbL/NvLtI5s2tcvmo7lwinJ7ksau1wdtnS\nMq+31zfA37PRES6VFbJ8/Wt1IvnmuLy5VuPbq5cjLcrafN51e3dS2+6d4XU8Mcsl3bML4Zhoxlqe\nbaKbu/3/ACB0BEQ1fSHElY1+4SdEoij4hUgUBb8QiaLgFyJRFPxCJEpPC3hmc4bhMZIZR6QLABib\nyoYNA7wI45l5XhB0PdLuKlfgxRvZtHaDZxA2WtyPc1Uuew1EstjWK1yaq66HC3jWIz62IjZ3svYA\nyiuRdl3D4UKow8O82Gm1yrd35ixfq8FBnl1omfD1zZpcJi7keBHXPq5Io1Dga7Xvun3UVq2EfXns\nsRfonGdePh3e1nr3WX268guRKAp+IRJFwS9Eoij4hUgUBb8QiaLgFyJReir1mRlyxfAui8M81398\nMHyOylW5jJYv8eymlUjfNLT4+bBUnApPyfN9tWq8n12hn/uRz/H1yGa5xFnzsC/1Bpc3PZK5Z1wR\ng9e55Ngipnwkmw4FLm8uL3Gpr1rn/elGRsPSbY5IgACQiax9BVxKmz+zSm1LkQzO1bVwlubf/+wl\nvi+iiq7XJfUJIS6Agl+IRFHwC5EoCn4hEkXBL0SiKPiFSJSeSn3ttqHMCiBmB+m8wYGwbpQvcR1q\nIJJ+NTLCpbnyCu8lV14JF1QsVyJZfevcNlTgBTCLpC8gADRrXOLM5cLn80LkNJ/v49loZnxif6QQ\naoaYmi0uRRVKkR6Ko1zeXFzkEtsqkT6Hx/naVyI9A185zguyvvTsCWqbHufZotN7yGvL8ON0ghQ0\nnV/lsuc7Nt/1M4UQ7ysU/EIkioJfiERR8AuRKAp+IRLlgnf7zawI4DEAfZ3nf9/dv2Bm1wD4DoBx\nAL8A8Gl3j7bhrdeB2dfDttoyvzs/NBm+Q1wsRRI6uHiA8XH+sstrvI7c8nLYtnSWJ4Is8ZvDyLb5\nXfa2cyWj1eIKAtphW+wsbxme2JPN8bWqRpKgnNzUz5M2XgDQrPCWYq1Ifb9WJFlouRyex7p4AcBi\nRPE5fpS/octn16itvsZ3uHMk3Mrrpqtn6Bzm4itvrtA5m+nmyl8D8Bvufis22nHfY2Z3AvgygK+5\n+/UAlgB8puu9CiG2nQsGv2/wVofKfOefA/gNAN/vjD8E4OOXxUMhxGWhq+/8ZpbtdOg9DeAnAF4F\nsOz+/z/czQLgn1GEEFccXQW/u7fc/QCAPQDuAHBT6GmhuWZ2yMyOmNmRc2Ve/EEI0Vve1d1+d18G\n8DMAdwIYNbO37gbtAXCKzDns7gfd/eDIYKTjgRCip1ww+M1s0sxGO49LAP49gBcB/BTA73aedh+A\nH18uJ4UQl55uEnt2AXjIzLLYOFl8z93/xsxeAPAdM/svAJ4C8MCFNuSWQys/EbQ1CgfpvFo7nMiS\naYZbUwFAcYTLV6OT/BPIWIYnnoxXwokWy4u8vdPyGS7nVdf48reaXD6E83N2uxn2cb3Kv3IVCpF6\ngTnu/+o6Tzypkq94+YgaPJQJJ6sAQDvDJaxGg69j30BYMi3meb3A0QL38VqMUtsHb+Vtw2685VZq\n23fddcHxO+7k8ubsqXJw/B9f5TGxmQsGv7s/A+C2wPgxbHz/F0K8B9Ev/IRIFAW/EImi4BciURT8\nQiSKgl+IRDGPZI9d8p2ZLQB4K69vAkD3usTlQ368Hfnxdt5rflzt7pPdbLCnwf+2HZsdcXcu7ssP\n+SE/Lqsf+tgvRKIo+IVIlO0M/sPbuO/zkR9vR368nfetH9v2nV8Isb3oY78QibItwW9m95jZv5jZ\nUTO7fzt86Phx3MyeNbOnzexID/f7oJmdNrPnzhsbN7OfmNkrnb9j2+THF83sZGdNnjazj/XAj71m\n9lMze9HMnjezP+mM93RNIn70dE3MrGhm/2xmv+z48Z8749eY2eOd9fiumUVSP7vA3Xv6D0AWG2XA\nrgVQAPBLADf32o+OL8cBTGzDfn8dwO0Anjtv7L8CuL/z+H4AX94mP74I4M96vB67ANzeeTwE4GUA\nN/d6TSJ+9HRNABiAwc7jPIDHsVFA53sAPtkZ/x8A/mgr+9mOK/8dAI66+zHfKPX9HQD3boMf24a7\nPwZgc53qe7FRCBXoUUFU4kfPcfc5d/9F5/EqNorFzKDHaxLxo6f4Bpe9aO52BP8MgPPbmW5n8U8H\n8Hdm9qSZHdomH95i2t3ngI2DEMDUNvryWTN7pvO14LJ//TgfM9uHjfoRj2Mb12STH0CP16QXRXO3\nI/hDJXa2S3K4291vB/BbAP7YzH59m/y4kvgGgP3Y6NEwB+ArvdqxmQ0C+AGAz7l7990nLr8fPV8T\n30LR3G7ZjuCfBbD3vP/T4p+XG3c/1fl7GsCPsL2ViebNbBcAdP6e3g4n3H2+c+C1AXwTPVoTM8tj\nI+C+5e4/7Az3fE1CfmzXmnT2/a6L5nbLdgT/EwCu79y5LAD4JICHe+2EmQ2Y2dBbjwH8JoDn4rMu\nKw9joxAqsI0FUd8Ktg6fQA/WxMwMGzUgX3T3r55n6umaMD96vSY9K5rbqzuYm+5mfgwbd1JfBfDn\n2+TDtdhQGn4J4Ple+gHg29j4+NjAxiehzwDYAeBRAK90/o5vkx//C8CzAJ7BRvDt6oEfv4aNj7DP\nAHi68+9jvV6TiB89XRMAt2CjKO4z2DjR/Kfzjtl/BnAUwP8B0LeV/egXfkIkin7hJ0SiKPiFSBQF\nvxCJouAXIlEU/EIkioJfiERR8AuRKAp+IRLl/wHCOW2RBgdIrQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x15ae4808780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the 0-th image (a frog)\n",
    "%matplotlib inline\n",
    "img = Image.fromarray(X_train[0,:,:,:])\n",
    "plt.imshow(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ham/Spam Texts Dataset (UCI)\n",
    "\n",
    "We will use another UCI ML Repository dataset called the SMS Spam Collection.  You can read about it here: https://archive.ics.uci.edu/ml/datasets/SMS+Spam+Collection .  As a sidenote about common terms, when predicting if a data point represents 'spam' (or unwanted advertisement), the alternative is called 'ham' (or useful information).\n",
    "\n",
    "This is a great dataset for predicting a binary outcome (spam/ham) from a textual input.  This will be very useful for short text sequences for Natural Language Processing (Ch 7) and Recurrent Neural Networks (Ch 9)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5574\n",
      "{'spam', 'ham'}\n",
      "Ok lar... Joking wif u oni...\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import io\n",
    "from zipfile import ZipFile\n",
    "\n",
    "# Get/read zip file\n",
    "zip_url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/00228/smsspamcollection.zip'\n",
    "r = requests.get(zip_url)\n",
    "z = ZipFile(io.BytesIO(r.content))\n",
    "file = z.read('SMSSpamCollection')\n",
    "# Format Data\n",
    "text_data = file.decode()\n",
    "text_data = text_data.encode('ascii',errors='ignore')\n",
    "text_data = text_data.decode().split('\\n')\n",
    "text_data = [x.split('\\t') for x in text_data if len(x)>=1]\n",
    "[text_data_target, text_data_train] = [list(x) for x in zip(*text_data)]\n",
    "print(len(text_data_train))\n",
    "print(set(text_data_target))\n",
    "print(text_data_train[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Movie Review Data (Cornell)\n",
    "\n",
    "The Movie Review database, collected by Bo Pang and Lillian Lee (researchers at Cornell), serves as a great dataset to use for predicting a numerical number from textual inputs.\n",
    "\n",
    "You can read more about the dataset and papers using it here:\n",
    "https://www.cs.cornell.edu/people/pabo/movie-review-data/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5331\n",
      "5331\n",
      "simplistic , silly and tedious . \n",
      "\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import io\n",
    "import tarfile\n",
    "\n",
    "movie_data_url = 'http://www.cs.cornell.edu/people/pabo/movie-review-data/rt-polaritydata.tar.gz'\n",
    "r = requests.get(movie_data_url)\n",
    "# Stream data into temp object\n",
    "stream_data = io.BytesIO(r.content)\n",
    "tmp = io.BytesIO()\n",
    "while True:\n",
    "    s = stream_data.read(16384)\n",
    "    if not s:  \n",
    "        break\n",
    "    tmp.write(s)\n",
    "stream_data.close()\n",
    "tmp.seek(0)\n",
    "# Extract tar file\n",
    "tar_file = tarfile.open(fileobj=tmp, mode=\"r:gz\")\n",
    "pos = tar_file.extractfile('rt-polaritydata/rt-polarity.pos')\n",
    "neg = tar_file.extractfile('rt-polaritydata/rt-polarity.neg')\n",
    "# Save pos/neg reviews\n",
    "pos_data = []\n",
    "for line in pos:\n",
    "    pos_data.append(line.decode('ISO-8859-1').encode('ascii',errors='ignore').decode())\n",
    "neg_data = []\n",
    "for line in neg:\n",
    "    neg_data.append(line.decode('ISO-8859-1').encode('ascii',errors='ignore').decode())\n",
    "tar_file.close()\n",
    "\n",
    "print(len(pos_data))\n",
    "print(len(neg_data))\n",
    "print(neg_data[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Complete Works of William Shakespeare (Gutenberg Project)\n",
    "\n",
    "For training a TensorFlow Model to create text, we will train it on the complete works of William Shakespeare.  This can be accessed through the good work of the Gutenberg Project.  The Gutenberg Project frees many non-copyright books by making them accessible for free from the hard work of volunteers.\n",
    "\n",
    "You can read more about the Shakespeare works here:\n",
    "http://www.gutenberg.org/ebooks/100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5582212\n"
     ]
    }
   ],
   "source": [
    "# The Works of Shakespeare Data\n",
    "import requests\n",
    "\n",
    "shakespeare_url = 'http://www.gutenberg.org/cache/epub/100/pg100.txt'\n",
    "# Get Shakespeare text\n",
    "response = requests.get(shakespeare_url)\n",
    "shakespeare_file = response.content\n",
    "# Decode binary into string\n",
    "shakespeare_text = shakespeare_file.decode('utf-8')\n",
    "# Drop first few descriptive paragraphs.\n",
    "shakespeare_text = shakespeare_text[7675:]\n",
    "print(len(shakespeare_text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## English-German Sentence Translation Database (Manythings/Tatoeba)\n",
    "\n",
    "The Tatoeba Project is also run by volunteers and is set to make the most bilingual sentence translations available between many different languages.  `Manythings.org` compiles the data and makes it accessible.\n",
    "\n",
    "http://www.manythings.org/corpus/about.html#info\n",
    "\n",
    "More bilingual sentence pairs: http://www.manythings.org/bilingual/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "147788\n",
      "147788\n",
      "['I won!', 'Ich hab gewonnen!']\n"
     ]
    }
   ],
   "source": [
    "# English-German Sentence Translation Data\n",
    "import requests\n",
    "import io\n",
    "from zipfile import ZipFile\n",
    "sentence_url = 'http://www.manythings.org/anki/deu-eng.zip'\n",
    "r = requests.get(sentence_url)\n",
    "z = ZipFile(io.BytesIO(r.content))\n",
    "file = z.read('deu.txt')\n",
    "# Format Data\n",
    "eng_ger_data = file.decode()\n",
    "eng_ger_data = eng_ger_data.encode('ascii',errors='ignore')\n",
    "eng_ger_data = eng_ger_data.decode().split('\\n')\n",
    "eng_ger_data = [x.split('\\t') for x in eng_ger_data if len(x)>=1]\n",
    "[english_sentence, german_sentence] = [list(x) for x in zip(*eng_ger_data)]\n",
    "print(len(english_sentence))\n",
    "print(len(german_sentence))\n",
    "print(eng_ger_data[10])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
